---
title: Spans
subtitle: Add router lifecycle context to traces
description: Use spans to add contextual information from the Apollo GraphOS Router or Apollo Router Core to traces displayed by your application performance monitors (APM).
context:
  - telemetry
redirectFrom:
  - /graphos/routing/observability/telemetry/instrumentation/spans
---

import RouterServices from '../../../../../shared/router-lifecycle-services.mdx';

A **span** captures contextual information about requests and responses as they're processed through the [router's request lifecycle (pipeline)](/graphos/routing/request-lifecycle). The information from spans can be used when displaying traces in your application performance monitors (APM).

## Spans configuration

### Router request lifecycle services

<RouterServices />

The `router`, `supergraph`, `subgraph`, `connector`, and `http_client` sections are used to define custom span configuration for each service:

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      router: # highlight-line
        attributes: {}
          # ...     
      supergraph: # highlight-line
        attributes: {}
          # ...
      subgraph: # highlight-line
        attributes: {}
          # ...      
      connector: # highlight-line
        attributes: {}
          # ...      
      http_client: # highlight-line
        attributes: {}
          # ...      
```

### `attributes`

Spans may have attributes attached to them from the router pipeline. These attributes are used to filter and group spans in your APM.

Attributes may be drawn from [standard attributes](/router/configuration/telemetry/instrumentation/standard-attributes) or [selectors](/router/configuration/telemetry/instrumentation/selectors). 

<PremiumFeature>

Granular customization of attributes on spans requires a [GraphOS plan](https://www.apollographql.com/pricing#observability).

</PremiumFeature>

The attributes that are available depend on the service of the pipeline.

```yaml title="desc.router.yaml"
telemetry:
  instrumentation:
    spans:
      router:
        attributes:
          # Standard attributes
          http.response.status_code: true
          # Custom attributes
          "my_attribute":
            response_header: "x-my-header"
```

You can also have [conditions](/router/configuration/telemetry/instrumentation/conditions) on custom attributes using [selectors](/router/configuration/telemetry/instrumentation/selectors). You can only have conditions on a selector at the same execution level.
Example you can't have a condition on `response_header` if you want to set an attribute from `request_header`.

```yaml title="desc.router.yaml"
telemetry:
  instrumentation:
    spans: 
      router: 
        attributes:    
          on_error: 
            response_status: reason
            condition:
              not:
                eq:
                - response_status: code
                - 200
```

### `default_attribute_requirement_level`

The `default_attribute_requirement_level` option sets the default attributes to attach to spans, as defined by [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/otel/common/attribute-requirement-level/).

Valid values:

* `required` (default, Apollo recommended)
* `recommended` (experimental attributes from OpenTelemetry's development-status conventions)

<Note>

Apollo recommends using `required`, rather than `recommended`. Using `recommended` includes experimental attributes from OpenTelemetry's [development-status GraphQL semantic conventions](https://opentelemetry.io/docs/specs/semconv/graphql/graphql-spans/), such as `graphql.document` and `subgraph.graphql.document`. These attributes can create high cardinality and may contain sensitive information. See the [standard attributes documentation](/router/configuration/telemetry/instrumentation/standard-attributes) for details.

</Note>

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      # Set the default requirement level
      default_attribute_requirement_level: required #highlight-line
```

Attributes can be configured individually, so that `required` attributes can be overridden or disabled. For example, `http.response.status_code` is set individually to override the standard value:

```yaml title="desc.router.yaml"
telemetry:
  instrumentation:
    spans:
      # Set the default requirement level
      default_attribute_requirement_level: required
      router:
        attributes:
          # Disable standard attribute
          http.response.status_code: false
```

<Note>

The attributes that the OpenTelemetry spec defines as `opt-in` must be configured individually.

</Note>

### `mode` 

The `mode` option enables the router spans to either use legacy attributes in the router, or those defined in the OpenTelemetry specification.

Valid values:

* `spec_compliant` (default)
* `deprecated`

#### `spec_compliant`

This mode is the default and follows the OpenTelemetry spec.

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      mode: spec_compliant
```

#### `deprecated`

This mode follows the previous behavior which is deprecated. The performance of this mode is significantly worse because many more attributes are added to spans, including attributes that do not follow OpenTelemetry conventions.

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      mode: deprecated
```

<Note>

The `mode` option will eventually be removed in a future release. 

</Note>

## Span status

By default spans are marked in error only if the http status code is different than 200. If you want to mark a span in error for other reason you can override the `otel.status_code` attribute which is responsible to mark a span in error or not.
If it's in error then `otel.status_code` = `error`, if not it will be `ok`.

## Naming

By default, we will use a span naming convention that aligns with the current [semantic conventions for GraphQL server in OpenTelemetry](https://opentelemetry.io/docs/specs/semconv/graphql/graphql-spans/) which means the root span name 
must be of format `<graphql.operation.type> <graphql.operation.name>` provided that `graphql.operation.type` and `graphql.operation.name` are available. 

If you want to change the name of spans we're creating for each services you can override this value by setting the `otel.name` attribute using any selectors you want.

Here is an example if you want to mark the `router` and `supergraph` span in error if you have a graphql error in the payload and you want to enforce the `router` span name to be `graphql_router`.

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      router:
        attributes:
          otel.name:
            static: graphql_router # Override the span name to graphql_router 
          otel.status_code:
            static: error
            condition:
              eq:
              - true
              - on_graphql_error: true
      supergraph:
        attributes:
          otel.status_code:
            static: error
            condition:
              exists:
                response_errors: $[0].extensions.code # Here is an example to get the first error code, `on_graphql_error` is also available for supergraph
```

## Span configuration example

An example configuration of `telemetry.spans` in `router.yaml` sets both standard and custom attributes for the router service:

```yaml title="router.yaml"
telemetry:
  instrumentation:
    spans:
      default_attribute_requirement_level: required
      mode: spec_compliant
      router:
        attributes:
          # Standard attributes (http)
          dd.trace_id: false
          http.request.body.size: false
          http.response.body.size: false
          http.request.method: false
          # ...
  
          # Conditional custom attribute
          otel.status_description: # You can conditionally put a status description attribute on your span if it respect the condition
            static: "there was an error"
            condition: # http response status code != 200 or it contains a graphql error in the payload
              any:
              - not:
                  eq:
                  - response_status: code
                  - 200
              - eq:
                - on_graphql_error
                - true
          # Custom attributes
          "acme.custom_1":
            trace_id: datadog
          "acme.custom_2":
            response_header: "X-CUSTOM2"
            default: "unknown"
          "acme.custom_3":
            env: "ENV_VAR"
          "static_attribute":
            static: "my_static_value"
            # ...
          
      supergraph:
        attributes: {}
          # ...
      subgraph:
        attributes: {}
          # ...      
      connector:
        attributes: {}
          # ...      
      http_client:
        attributes: {}
          # ...      
```

## Spans configuration reference

| Option                                | Values                                                                    | Default                        | Description                              |
|---------------------------------------|---------------------------------------------------------------------------|--------------------------------|------------------------------------------|
| `<attribute-name>`                    |                                                                           |                                | The name of the custom attribute.        |
| `attributes`                          | [standard attributes](/router/configuration/telemetry/instrumentation/standard-attributes)\|[selectors](/router/configuration/telemetry/instrumentation/selectors)    |                                | The attributes of the span.              |
| `condition`                           | [conditions](/router/configuration/telemetry/instrumentation/conditions)                                                |                                | The condition for adding a custom attribute. |
| `default_attribute_requirement_level` | `required`\|`recommended`                                                 | `required`                     | The default attribute requirement level. |
| `mode`                                | `spec_compliant` \| `deprecated`                                          | `spec_compliant`                   | The attributes of the span.              |

